Pen-based devices, including Tablet PCs, personal digital assistants (PDAs), and advanced smart phones, are increasingly important forms of computing and commutation. One of the challenges to make these devices usable is the lack of physical input devices such as the mouse and the desktop keyboard. In particular, conventional pen-based devices lack convenient methods for issuing commands to operating systems and applications on these devices. On desktop computers, commands are accessible through graphical menus. Frequent commands, such as copy and paste commands, are also accessible through keyboard shortcuts. Shortcuts are faster and less attention demanding than graphical menus. A user does not have to look for the menus whose locations vary from application to application. Certain shortcuts tend to be universal across applications. For example, Control-C typically executes a copy command. Similarly, Control-B typically executes a bold command. A user can reliably count on these basic operations for various software applications.
A basic method of issuing commands on a pen-based computer is the same as on a desktop PC: linear and hierarchical pull-down menus. Pull-down menus are more problematic on a pen-based computer for a number of reasons. The pen (stylus) and the stylus holding hand of the user often obscure the very items on the pull-down menu the user needs to find and select. Moreover, pen motion on a screen is one-to-one in scale. In contrast, other pointing devices comprise a rate-accelerated control-to-display gain (e.g., a power-mouse), so that a user does not have to move the pointing device over a large distance to reach a far-away menu. Furthermore, moving a pen on the screen is usually not as well supported as moving a mouse on a desktop. Consequently, an alternative shortcut approach to the graphical menu items are even more desirable for pen-based devices.
Several conventional methods enhance command selection capabilities on pen-based computers and devices. One conventional method demonstrates a technique that makes linear menu traversal easier when navigating through sub-menus [reference is made to Kobayashi, M., et. al., “Considering the Direction of Cursor Movement for Efficient Traversal of Cascading Menus”, Proc. UIST 2003, pp. 91-94]. Another conventional method comprises a technique called Hotbox that combines linear, radial and pop-up menus to create a Graphical User Interface (GUI) that can handle over 1,000 commands for the a graphical modeling application [reference is made to Kurtenbach, G., et. al., “The Hotbox: Efficient Access to a Large Number of Menu-items”, Proc. CHI 1999, pp. 231-237.]
Other conventional methods utilize pie menus as an alternative to pull-down linear menus [reference is made to Callahan, J., et. al., “An Empirical Comparison of Pie vs. Linear Menus”, Proc. CHI 1988, pp. 95-100]. Marking menus further improve pie menus [reference is made to Kurtenbach, G., et. al., “The Limits of Expert Performance Using Hierarchic Marking Menus”, Proc. CHI 1993, pp. 482-487]. Marking menus are pie menus augmented with a gesture recognizer. Novice users select items in the pie menu structure as if using a regular pie menu (albeit with delayed display). Over time, users learn the angular gestures for selecting an command. This allows expert users to quickly flick the gesture of the command without the need to visually traverse a pie menu hierarchy. To encourage users to learn the fast mode of gesturing commands instead of using slower visual feedback-based navigation, marking menus do not “pop-up” until after a time delay.
However, some marking menu selections are ambiguous when marks are articulated independent of scale. Furthermore, deeply nested menu selections result in complex gestures that cannot be articulated on a small space. One conventional method uses consecutive simple marks instead of compound marks in marking menus [reference is made to Zhao, S., et. al., “Simple vs. Compound Mark Hierarchical Marking Menus”, Proc. UIST 2004, pp. 33-42]. Simple marks are unambiguous and require only a small space to articulate. Simple marks can be articulated approximately as fast as compound marks. However, users may not be able to memorize a sequence of disconnected simple marks as fast and as easily as compound marks that can be perceived and remembered as a whole. Another conventional system uses FlowMenu, a variant of the marking menu [reference is made for example, to Guimbretière, F., et. al., “FlowMenu: Combining Command, Text and Data Entry”, Proc. UIST 2000, pp. 213-216]. With FlowMenu, a user can, for example, choose to zoom with a numeric value and enter the zoom value in one holistic gesture. However, gestures required by FlowMenu are long and complicated.
Other conventional systems use free-form pen gestures as command articulation. These conventional systems come in many forms and variants such as, for example, the Rubine gesture recognizer [reference is made to Rubine, D., “Specifying Gestures by Example”, Proc. SIGGRAPH 1991, pp. 329-337]. Free-form pen gestures are often “arbitrary” gestures that are used to denote different actions in a user interface. In general, free form gestures are limited in capacity (the number a user can remember and reproduce) due to their arbitrary definition to the user.
Each conventional pen-based command selection technique has advantages and disadvantages. Users are familiar with linear menus and they are backwards compatible with virtually all existing desktop and handheld programs. Marking menus have an advantage in applications where the user frequently needs rapid access to a limited number of commands. Special techniques such as the Hotbox can be used when an application needs to provide access to over 1,000 commands. Free-form pen gestures are most advantageous in application domains that have a strong convention, such as copy editing, or in places where gestures can be highly metaphorical, such as crossing a series of words to cause the words to be deleted.
Although some of these conventional solutions are adequate for their intended purpose, it would be desirable to have a pen-based system and an associated method for issuing commands that is similar to the role of keyboard shortcuts in desktop computers while taking advantage of the fluidity of the pen. Such a system and method should be easy to learn, obvious to the user, fast to execute, and easily applicable to software applications without modification. The need for such a solution has heretofore remained unsatisfied.